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Enough  reviews  for  you???? 


♦  Review 

♦  Management  Review 

♦  Technical  Review 

♦  Inspection 

♦  Peer  Review 

♦  Walk-Through 

♦  Audit 


♦  Code  Review 

♦  Design  Review 

♦  Formal  Qualification 
Review 

♦  Requirements 
Review 

♦  Test  Readiness 
Review 
Functional 
Configuration  Audit 

♦  Physical 
Configuration  Audit 

♦  Etc. 


♦  “Skim”  Review 

♦  Disciplined  Document  Review 

♦  Desk  Check 

♦  Personal  Document  Review 

♦  Personal  Code  Review 
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What  reviews  give  you 

♦  Direct  Benefits 

-  Improved  code  quality 

-  Fewer  Defects 

-  Improved  communication  about  code  content 

-  Education  of  new/junior  developers 

♦  Indirect  benefits 

-  Shorter  and  more  effective  testing 

-  Less  maintenance 

-  Improved  customer  satisfaction 

-  More  maintainable  code 
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Quality  is  the  goal 


♦  Quality  is  NOT  free 

♦  “Cost”  of  Quality  includes 

-  Review  costs 

-  T ests  cost 

-  All  defect  prevention  costs  (training) 

♦  Savings  from  Quality  include 

-  Decreased  costs  of  product  failure 

^  Help  Desk 

*-  Customer  defect  repair 

-  Shorter  test  cost 

-  Shorter  development  time 
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Return  on  Investment 


♦  Boeing  -  33:1  savings  from  reviews 

♦  HP  -  10:1  saving  $21  million  a  year 

♦  Space  Shuttle  -  $1  if  error  found  in 
inspection,  $13  during  test,  $92  after  delivery 

♦  IBM  -  each  hour  of  inspection  saved  20  hours 
of  testing,  and  82  hours  of  rework  (for  each 
error  that  would  have  made  it  to  delivery) 

♦  AT&T  -  22:1  savings  if  errors  found  early, 
reduced  cost  of  finding  errors  by  10:1 
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More  savings 

♦  Maintenance  costs  are  typically  50%  less 
(values  of  90%  have  been  reported) 

♦  Litton  Data  Systems  -  3%  increase  in  costs 
due  to  inspections,  number  of  errors  found 
during  system  and  integration  testing  dropped 
30% 
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Reviews  vs.  Testing 


♦  Testing  is  a  discrete  activity,  reviews  should 
be  continuous 

♦  Each  testing  stage  only  removes  about  35% 
of  errors  present 

♦  GOOD  Reviews  and  Inspections  typically 
remove  50% 

♦  Testing  can  give  poor  code  coverage,  and  will 
always  give  poor  coverage  of  documentation 
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What  can  be  reviewed? 


♦  ??  (fill  in  the  blanks) 
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What  can  be  reviewed? 


♦  ??  (fill  in  the  blanks) 


What  can’t  be  reviewed? 
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Management  Involvement  is  limited 

♦  Measurement  dysfunction  -  when  managers 
use  review  data  to  evaluate.  This  produces 
inconsistent  results  and  bizarre  behavior. 

♦  Leads  to  inaccurate  data,  invalid  reviews,  and 
the  use  of  reviews  to  grind  “personal”  issues. 

♦  Management  involvement  should  be  limited 
to  “edited”  and  “sanitized”  summarization  of 
the  final  results 
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Management  Commitment 

♦  Provide  resources  (time  and  space) 

♦  Setting  policies  and  goals 

♦  Maintaining  reviews  even  when  under  a  time 
crunch 

♦  Require  schedules  to  include  review  time 

♦  Providing  training 

♦  Not  using  results  to  evaluate 

♦  Holding  people  accountable  for  participation 
and  contributions 
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Management  Commitment  (cont.) 

♦  Rewarding  early  adopters 

♦  Running  interference  with  challengers 

♦  Respecting  review  team’s  appraisal 

♦  Asking  for  status  reports,  showing  how  the 
program  is  working,  what  it  costs,  and  the 
benefits  (and  deficits) 
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Consequences  of  Misapplication  of 
Inspection  Data 

♦  Developers  might  not  submit  products 

♦  Developers  might  not  agree  to  review  peer’s 
work 

♦  Defects  brought  up  after  the  review,  not 
during 

♦  Pre-reviews  to  prepare 

♦  Too  much  debate  on  what  is  a  defect 

♦  Review  of  small  products  -  wasting  time 
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Ego-less  programming 


♦  Need  to  stress  the  benefits  of  reviews  to  all 
levels  of  management 

-  Less  time  in  rework 

-  Increased  productivity 

-  Education  and  learning 

-  Better  able  to  meet  deadlines 

-  Better  risk  management 

-  Not  “extra  time”,  but  reallocation  of  effort 
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Reviews  are  NOT  milestones 

♦  Milestones  are  a  “time” 

♦  Reviews  are  a  “process” 

♦  Milestones  occur  AFTER  a  review,  and 
involve  a  go/no-go  decision 
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Principles  for  a  review 

1 .  Check  egos  at  the  door 

2.  Keep  the  review  team  small 

3.  Find  problems,  don’t  solve  them 

4.  Limit  review  time 

5.  Require  preparation 
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Peer  Review  Spectrum 


♦  Inspection 

♦  Team  Review 

♦  Walkthrough 

♦  Pair  Programming 

♦  Peer  Deskcheck 

♦  Ad  Hoc 


Most  formal 

▲ 


▼ 

Least  formal 
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Typical  Activities 


REVIEW 

TYPE 

Planning 

Preparation 

Meeting 

Corrections 

Verification 

Inspection 

Yes 

Yes 

Yes 

Yes 

Yes 

Team  Review 

Yes 

Yes 

Yes 

Yes 

No 

Walkthrough 

Yes 

No 

Yes 

Yes 

No 

Pair 

Programming 

Yes 

No 

N/A 

Yes 

Yes 

Peer 

Deskcheck 

No 

Yes 

Maybe 

Yes 

No 

Ad  Hoc 

No 

No 

Yes 

Yes 

No 

Individual 

No 

No 

No 

Yes 

Always 
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Which  type  of  review  for  you? 


♦  Depends  upon 

-  Criticality  of  application 

-  Skill  of  individual  reviewer 

-  Needs  of  the  organization 

-  Maturity  of  the  organization 
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Review 

Objective 

Inspection 

Team  Review 

Walkthrough 

Pair 

Programming 

Peer 

Deskcheck 

Passaround 

Find  defects 

X 

X 

X 

X 

X 

X 

Conformance 
to  specs 

X 

X 

X 

X 

Verify 

complete  and 
correct 

X 

X 

Assess 
under- 
standability 
and  main¬ 
tainability 

X 

X 

X 

X 

Demonstrate 

quality 

X 

Collect  data 
for 

improvement 

X 

X 
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Review 

Objective 

Inspection 

Team  Review 

Walkthrouah 

Pair 

Proarammina 

Peer 

Deskcheck 

Passaround 

Measure 

quality 

X 

Education  of 
team 
members 

X 

X 

X 

X 

Reach 

consensus  on 
approach 

X 

X 

X 

X? 

Ensure 
changes  of 
fixes  made 
correctly 

X 

X 

X 

Explore 

alternative 

approaches 

X 

X 

Simulate 
execution  of  a 
program 

X 

Minimize 
review  cost 

X 
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Common  Misconception 

♦  Peer  reviews  are  a  luxury 

♦  TRUTH:  Peer  reviews,  when  intelligently 
applied,  shorten  development  and  testing.  In 
fact,  some  testing  steps  may  be  skipped  (or 
will  be  so  small  they  are  almost  a  formality) 
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How  fast  to  review 


♦  Studies  show  that  200  LOC/hour  is  close  to 
optimal 

-  More,  and  you  miss  errors 


-  Less,  and  you  get  diminishing  returns 

♦  With  200  LOC/hour,  defects  will  be  reduced 
to  around  20  per  1000  LOC 
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Rules  for  reviews 


♦  Schedule  no  less  than  a  week  in  advance,  to 
give  participants  time  to  prepare 

♦  No  more  than  one  inspection  per  day  for  any 
one  participant  (including  the  moderator) 

♦  No  “lunch”  inspections 

♦  No  “3  PM  Friday”  inspections 

♦  Coffee  and  donuts  are  a  necessity 

♦  Have  a  time  limit  -  and  STICK  TO  IT!!  End 
when  the  time  is  up 
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Before  the  review  -  perform  “Skim  Review” 


♦  Brief  one-time  reading  (similar  to  reading  a 

novel) 

♦  Guidelines  for  “skim  review” 

-  Don’t  depend  on  ad-hoc,  skim  reviews  to  find 
all  (or  even  most  of)  the  defects 

-  Use  them  to  overview  document 

-  Use  them  to  check  that  entrance  criteria  for 
review  have  been  met  (e.g.,  not  more  than  3 
major  defects  found  in  10  minutes) 
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During  any  structured  review 

♦  Have  recorder!!! 

♦  Have  a  recorder  who  knows  how  to 
record!!! 

♦  Use  semi-formal  &  formal  documents  to 
record  errors  (location,  side  effects,  any 
other  specifics) 

♦  Use  the  same  documentation  to  provide 
accountability  and  reduce  need  for  follow¬ 
up  (although  spot-checking  of  follow-up  is 
HIGHLY  recommended) 
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Seven  “Truths”  about  Reviews 

♦  Peer  reviews  can  take  many  forms 

♦  Inspections  are  a  software  industry  best 
practice 

♦  There  is  no  one  true  inspection  method 

♦  Peer  reviews  complement  testing 

♦  Peer  reviews  are  both  technical  and  social 
activities 

♦  Managers  can  make  or  break  a  review 
program 

♦  A  peer  review  program  doesn’t  run  itself 

♦ 

This  slide  and  the  next  from  Karl  Wiegers’  Book 
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Comparison  of  methods 


Element 

Fagan  Method 

Gilb/Graham  Method 

Process  Steps 

•Planning 

•Planning 

•Overview 

•Kickoff  Meeting 

•Preparation 

•Individual  Checking 

•Inspection  Meeting 

•Logging  Meeting 

•Rework 

•Editing 

•Follow-up 

•Follow-up 

•Causal  Analysis 

•Process  Brainstorming  Meeting 

Roles 

•Author 

•Author 

•Moderator 

•Inspection  Leader 

•Reader 

•Scribe 

•Recorder 

•Inspector 

•Checker 

Defect-Detection 

•Defect  Checklists 

•Rule  Sets 

Techniques 

•Checklists 

Emphasis 

•Removing  Defects 

•Document  Quality 
•Measurement 
•Process  Improvement 
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Remember  -  to  make  reviews  work... 


♦  No  discipline  or  rigor  is  normally  associated  with  informal 
reviews,  so  effective  leaders  and  checklists  must  be  used 
to  achieve  useful  results 

♦  To  make  reviews  useful,  members  of  the  the  review  team 
must  be  objective 

-  Make  sure  that  some  members  of  the  review  team  have  different 
backgrounds 

-  Make  sure  that  some  members  of  the  review  team  have  no  direct 
involvement  with  the  product  being  reviewed 

-  Political  agendas  need  to  be  left  at  the  door 

♦  Make  sure  that  reviewers  understand  the  requirements 

-  If  necessary,  present  requirements  in  a  number  of  different  ways 

-  Simply  reading  the  requirements  documents  is  probably  insufficient 

-  The  brain  can  only  keep  so  many  requirements  “active” 
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Questions??? 
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